Modern electronic devices come in many forms. Personal modern electronic devices include smart watches, mobile phones, and notebook computers. Modern electronic devices deployed by corporations include server machines that power large data centers and cloud computing services, plus computing technology that is embedded in other devices, such as vehicles and manufacturing equipment. Further, internet-of-things (IoT) devices are modern electronic devices that employ processing capabilities to improve device interconnectedness. These IoT devices, which are proliferating in many areas of life, include thermostats, refrigerators, portable medical equipment, and so forth. Each of these kinds of electronic devices play some pivotal role in modern life.
What each of these electronic devices have in common is some kind of processor. Processors operate as the brains of electronic devices by implementing some functionality that has been encoded into a program that can be executed. To execute a program and thereby provide the desired device functionality, the processor uses memory to store the program code or the data on which the program operates. Thus, the processor and associated memory work together to execute programs.
One reason electronic devices are more common today is that modern electronic devices are so much more powerful than those of just a few years ago, not to mention ones from a few decades ago. One reason electronic devices are more powerful is that processors have become faster. Generally, the faster a processor can execute a program, the more interesting or beneficial is the functionality that the processor can provide. However, intrinsic processor speed alone does not determine an amount of functionality that an electronic device can provide because intrinsic processor speed alone does not control how quickly a processor can execute a program. The speed of memory associated with a processor, for example, also impacts program execution speed.
Accordingly, efforts have been made to increase the speed at which a processor can use memory to execute programs. One approach is to utilize a type of memory with relatively superior speed. Two example types of memory are dynamic random access memory (DRAM) and static random access memory (SRAM). On the one hand, the SRAM type is faster than the DRAM type. On the other hand, SRAM costs more because SRAM is both more complicated and occupies a larger chip area on a per-bit basis. Nevertheless, SRAM is often used for the memory that a processor accesses because superior speed can be worth the price- and chip-area premiums.
Another approach is to place a portion of the memory closer to the processor that is to access the memory and another portion further away. Having multiple levels of memory like this is called hierarchical memory. Two different levels of hierarchical memory are cache memory and main memory. The contents of a cache memory may be only part of the entire contents of the corresponding main memory. However, the cache memory is placed closer to the processor as compared to the main memory to reduce data access times. Moreover, with modern processors cache memory is typically integrated on a same integrated circuit (IC) chip as is the processor. This integration enables the processor to access the cache memory with significantly lower latencies.
In many modern processors, these two approaches are used in combination. In other words, a processor is provided a cache memory formed from SRAM cells. Such an SRAM cache memory can also be integrated with a processor on the same chip. Implementing these approaches does increase the speed at which a processor can use memory to execute programs and provide computing functionality. Unfortunately, even if these various approaches are implemented together, memory usage still causes a bottleneck for processing throughput. The time delays that occur because a processor is interacting with a slower memory continue to negatively impact the speed at which a processor can execute a program. Consequently, memory system deficiencies continue to hinder overall computing performance and limit the functionality that can be provided by modern electronic devices.